Real-Time Inverse Dynamics Learning for Musculoskeletal Robots based on Echo State Gaussian Process Regression

A challenging topic in articulated robots is the control of redundantly many degrees of freedom with artificial muscles. Actuation with these devices is difficult to solve because of nonlinearities, delays and unknown parameters such as friction. Machine learning methods can be used to learn control of these systems, but are faced with the additional problem that the size of the search space prohibits full exploration in reasonable time. We propose a novel method that is able to learn control of redundant robot arms with artificial muscles online from scratch using only the position of the end effector, without using any joint positions, accelerations or an analytical model of the system or the environment. To learn in real time, we use the so called online “goal babbling” method to effectively reduce the search space, a recurrent neural network to represent the state of the robot arm, and novel online Gaussian processes for regression. With our approach, we achieve good performance on trajectory tracking tasks for the end effector of two very challenging systems: a simulated 6 DOF redundant arm with artificial muscles, and a 7 DOF robot arm with McKibben pneumatic artificial muscles. We also show that the combination of techniques we propose results in significantly improved performance over using the individual techniques alone.